“Virtual reality” (VR) may sometimes refer to video output of animated digital content based on three-dimensional computer models, including, for example, various video game content, and animated film content. In this type of virtual reality, in some implementations a user can navigate through a three-dimensional (“3D”) environment generated based on the computer model, by controlling the position and orientation of a virtual camera that defines a viewpoint for two-dimensional (“2D”) scene that is displayed on a two-dimensional display screen. In other implementations, the virtual reality may be experienced more passively, without viewer interaction that changes the scene, as when watching a movie or the like.
More recently, “virtual reality” has been applied to various types of immersive video stereographic presentation techniques including, for example, stereographic virtual reality headsets. Headsets and other presentation methods immerse the user in a 3D scene. Lenses in the headset enable to user to focus on a lightweight split display screen mounted in the headset only inches from the user's eyes. Different sides of the split display show right and left stereographic views of video content, while the user's peripheral view is blocked. Thus, the headset enables the user to perceive 3D depth in the scene and feel immersed in the displayed content. In addition, the headset includes motion sensors that detect motion of the user's head, and adjust the video display accordingly. Thus, by turning his head to the side, the user can see the scene off to the side; by turning his head up or down, the user can look up or down in the scene. This responsiveness to head movement greatly enhances the immersive effect of content accessed via the headset. The user may be provided the impression of being placed inside the virtual reality scene. Virtual reality headsets may be configured as glasses, goggles, visors, or in some other form factor.
Augmented reality (AR) is related to virtual reality in that it can be implemented using similar glasses or headset as used for VR. In certain implementations, AR may be used to provide an immersive, augmented experience of real-world objects by overlaying a stereographic image over a view of the surrounding physical environment.
These immersive virtual reality headsets and other immersive technologies are especially useful for game play of various types, which involve user exploration of a modelled environment generated by a rendering engine as the user controls a virtual camera using head movement and/or other inputs. Immersive VR is generally designed to accommodate non-story telling applications, for example, video games of various types. The user experience provided by immersive VR is that of being immersed in an alternative reality. To provide this experience, the user needs to perceive a freedom of movement that is in some way analogous to human visual perception when interacting with reality. Content produced for VR can provide this experience using techniques for real-time rendering that have been developed for various types of video games. The content may include a three-dimensional computer model with defined boundaries and rules, configured for rendering as video output. This content can be enhanced by stereographic techniques to provide stereographic output, sometime referred to as “3D,” and associated with a VR application that manages the rendering process in response to movement of the VR headset, to produce a resulting VR experience. The user experience is very much like being placed inside a rendered video game. A similar process can be used for AR, but including a view of the user's physical environment in the immersive experience.
VR content, AR content, and video game content may also be used for storytelling, but applications to date have been limited by limitations on real-time rendering, among other things. Computer-generated stereographic animated content is well known, and often used in major motion pictures and other storytelling venues. Consumers have come to expect a very high rendering quality for such storytelling content that will not be possible to achieve by real-time rendering in the near future, if ever. Rendered content that produces realistic human actors requires a very high degree of animating skill and production effort that tends to limit this use of computer graphics to special effects. For these and other reasons, storytelling content for video game or VR environments tends to be short, simple, not of the highest rendering quality, and not the main focus of the content, which is focused on game play.
VR headsets and AR headsets have also been contemplated as an outlet for video content produced by shooting real sets, by providing production studio rendered video content, or some combination of such video content. However, the production of video content for immersive VR or AR consumption entails a completely different set of challenges and opportunities from real-time rendered content. Equipment and methods for producing VR or AR immersive content from captured or production studio rendered video content are subject to certain limitations. For example, in the absence of a computer model of the scene used for real-time rendering, it may be necessary to store and access a massive amount of video data to support a VR or AR immersive output. Such output may provide the user the experience of looking around a scene, but only from the single position from which the scene was shot, creating a “locked position” effect. For many types of video content, the ability to look all around a scene may tend to distract a user from the story that the director intends to convey, or even obstruct the relevant action, resulting in a less satisfying and focused experience for the user.
It would be desirable, therefore, to develop new hardware, production methods, output techniques, and other new technologies for video-sourced immersive VR and AR content and for use in combination with such content that overcomes these and other limitations of the prior art, and enhances enjoyment of video-captured content.